{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3f64620a-5c2d-4f93-8c92-a54dbdd672a0",
   "metadata": {},
   "source": [
    "# Quick Start for Segment Anything with Microscopy\n",
    "\n",
    "This notebook shows how to get quickly started with `µsam` (Segment Anything for Microscopy).\n",
    "\n",
    "Here, we show the very basic necessary functions to automatically segment a microscopy image using [µsam](https://github.com/computational-cell-analytics/micro-sam).  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18afa382-1267-4259-9581-87168bf71f43",
   "metadata": {},
   "source": [
    "## Running this notebook\n",
    "\n",
    "If you have an environment with `µsam` on your computer you can run this notebook in there. You can follow the [installation instructions](https://computational-cell-analytics.github.io/micro-sam/micro_sam.html#installation) to install it on your computer.\n",
    "\n",
    "You can also run this notebook in the cloud on [Kaggle Notebooks](https://www.kaggle.com/code/). This service offers free usage of a GPU to speed up running the code. The next collapsible section guides you through the installation for you if you are using it. You can skip them if you have `micro_sam` installed.\n",
    "\n",
    "<details>\n",
    "\n",
    "<summary><strong>Click here for more installation details.</strong></summary>\n",
    "\n",
    "## Install `condacolab` for Package Management\n",
    "\n",
    "**IMPORTANT**: This step is only necessary if you are running this notebook in Kaggle Notebooks (tested) / Google Colab. **Please skip these cells and go to `Importing the libraries` if you are running the notebook on your own computer with `micro_sam` already installed.**\n",
    "\n",
    "The first step for running the notebook on the above mentioned cloud platforms is to have a package manager to install `µsam` and other relevant packages.\n",
    "\n",
    "**NOTE**: Please run the next two blocks of code in new cells one after the other.\n",
    "\n",
    "# Step 1: Install `condacolab` on Kaggle Notebooks (tested) / Google Colab for installing conda and other related package managers.\n",
    "\n",
    "```bash\n",
    "!pip install -q condacolab\n",
    "import condacolab\n",
    "condacolab.install()\n",
    "```\n",
    "\n",
    "> **NOTE**: When you run the previous block of code, the kernel should restart automatically in Kaggle Notebooks (tested). Please proceed further without the need of any manual kernel restarts.\n",
    "\n",
    "# Step 2: Validate if `condacolab` was installed correctly.\n",
    "\n",
    "```bash\n",
    "import condacolab\n",
    "condacolab.check()\n",
    "```\n",
    "\n",
    "> If you see `✨🍰✨ Everything looks OK!` in the previous code block's output, you are good to proceed the next code block!\n",
    "\n",
    "## Checking system information\n",
    "\n",
    "```python\n",
    "# Check if we are running this notebook on Kaggle, Google Colab or local compute resources.\n",
    "import os\n",
    "current_spot = os.getcwd()\n",
    "\n",
    "if current_spot.startswith(\"/kaggle/working\"):\n",
    "    print(\"Kaggle says hi!\")\n",
    "    root_dir = \"/kaggle/working\"\n",
    "\n",
    "elif current_spot.startswith(\"/content\"):\n",
    "    print(\"Google Colab says hi!\")\n",
    "    print(\" NOTE: The scripts have not been tested on Google Colab, you might need to adapt the installations a bit.\")\n",
    "    root_dir = \"/content\"\n",
    "\n",
    "else:\n",
    "    msg = \"You are using a behind-the-scenes resource. Follow our installation instructions here:\"\n",
    "    msg += \" https://computational-cell-analytics.github.io/micro-sam/micro_sam.html#installation\"\n",
    "    print(msg)\n",
    "    root_dir = \"\"  # overwrite to set the root directory, where the data, checkpoints, and all relevant stuff will be stored\n",
    "```\n",
    "\n",
    "### Installation\n",
    "\n",
    "The next code block will install the `micro_sam` library on Kaggle Notebooks. **Please skip these cells and go to `Importing the libraries` if you are running the notebook on your own computer.**\n",
    "\n",
    "```bash\n",
    "# Install `micro-sam` and `stackview` (https://github.com/haesleinhuepf/stackview) from `conda-forge`.\n",
    "!mamba install -y -q -c conda-forge micro_sam stackview > /dev/null 2>&1\n",
    "!pip uninstall -y --quiet napari  # GUI backends are not supported in Kaggle / Google Colab, let's remove it to avoid errors.\n",
    "```\n",
    "\n",
    "> NOTE:\n",
    "Known Issues on **Kaggle Notebooks** (for the `Importing the libraries` section below):\n",
    "\n",
    "1. ```bash\n",
    "    2024-05-08 14:56:42.941557: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
    "    2024-05-08 14:56:42.941680: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
    "    2024-05-08 14:56:43.062385: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
    "    ```\n",
    "\n",
    "    - We have received this warning while testing this notebook on Kaggle. We suspect that the complains originate from the tensorflow suite. It does not lead to any issues while making use of the installed packages. You can proceed and ignore the warnings."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bbc620f",
   "metadata": {},
   "source": [
    "## Importing the libraries\n",
    "\n",
    "NOTE: For result visualization, we use [stackview](https://github.com/haesleinhuepf/stackview) (if you are running the notebook on your own computer, you can install `stackview` from `conda-forge` using `conda install -c conda-forge stackview` or using `pip`: `pip install stackview`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a00fdeac-2f97-4da5-8c32-46b51eb3d684",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-04-25T11:26:34.996024Z",
     "iopub.status.busy": "2025-04-25T11:26:34.995703Z",
     "iopub.status.idle": "2025-04-25T11:27:00.213327Z",
     "shell.execute_reply": "2025-04-25T11:27:00.212538Z",
     "shell.execute_reply.started": "2025-04-25T11:26:34.995998Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2025-04-25 11:26:45.843156: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
      "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n",
      "E0000 00:00:1745580406.038269     172 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
      "E0000 00:00:1745580406.093288     172 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n"
     ]
    }
   ],
   "source": [
    "import stackview\n",
    "\n",
    "from skimage.data import cells3d\n",
    "\n",
    "from micro_sam.automatic_segmentation import get_predictor_and_segmenter, automatic_instance_segmentation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "246e025f-e366-4795-9046-8aa0bfd91a25",
   "metadata": {},
   "source": [
    "We load an example image from the [scikit-image](https://scikit-image.org/docs/stable/api/skimage.data.html) library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a9bb2ea5-d6dd-4329-a9bc-45d21a3e8635",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-04-25T11:27:00.215603Z",
     "iopub.status.busy": "2025-04-25T11:27:00.214679Z",
     "iopub.status.idle": "2025-04-25T11:27:01.228148Z",
     "shell.execute_reply": "2025-04-25T11:27:01.227339Z",
     "shell.execute_reply.started": "2025-04-25T11:27:00.215580Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Downloading file 'data/cells3d.tif' from 'https://gitlab.com/scikit-image/data/-/raw/2cdc5ce89b334d28f06a58c9f0ca21aa6992a5ba/cells3d.tif' to '/root/.cache/scikit-image/0.25.2'.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<tr>\n",
       "<td>\n",
       "<img src=\"\"></img>\n",
       "</td>\n",
       "<td style=\"text-align: center; vertical-align: top;\">\n",
       "\n",
       "<table>\n",
       "<tr><td>shape</td><td>(256,&nbsp;256)</td></tr>\n",
       "<tr><td>dtype</td><td>uint16</td></tr>\n",
       "<tr><td>size</td><td>128.0 kB</td></tr>\n",
       "<tr><td>min</td><td>277</td></tr><tr><td>max</td><td>44092</td></tr>\n",
       "</table>\n",
       "<img src=\"\"></img>\n",
       "</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "StackViewNDArray([[4496, 5212, 6863, ..., 2917, 2680, 2642],\n",
       "                  [4533, 5146, 7555, ..., 2843, 2857, 2748],\n",
       "                  [4640, 6082, 8452, ..., 3372, 3039, 3128],\n",
       "                  ...,\n",
       "                  [1339, 1403, 1359, ..., 4458, 4314, 4795],\n",
       "                  [1473, 1560, 1622, ..., 3967, 4531, 4204],\n",
       "                  [1380, 1368, 1649, ..., 3091, 3558, 3682]], dtype=uint16)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "image = cells3d()[30, 0]\n",
    "\n",
    "# Visualize the image.\n",
    "stackview.insight(image)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b915aa3-ca50-4339-bf6a-866d35a87f2c",
   "metadata": {},
   "source": [
    "Loading a pre-trained micro-sam model and applying it to an image just takes two lines of python code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6542f7b7-d66b-46f7-93c8-e05d415cbada",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-04-25T11:29:01.982045Z",
     "iopub.status.busy": "2025-04-25T11:29:01.981742Z",
     "iopub.status.idle": "2025-04-25T11:29:34.505352Z",
     "shell.execute_reply": "2025-04-25T11:29:34.504538Z",
     "shell.execute_reply.started": "2025-04-25T11:29:01.982022Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Downloading file 'vit_b_lm' from 'https://uk1s3.embassy.ebi.ac.uk/public-datasets/bioimage.io/diplomatic-bug/1.1/files/vit_b.pt' to '/root/.cache/micro_sam/models'.\n",
      "100%|████████████████████████████████████████| 375M/375M [00:00<00:00, 407GB/s]\n",
      "Downloading file 'vit_b_lm_decoder' from 'https://uk1s3.embassy.ebi.ac.uk/public-datasets/bioimage.io/diplomatic-bug/1.1/files/vit_b_decoder.pt' to '/root/.cache/micro_sam/models'.\n",
      "100%|█████████████████████████████████████| 44.1M/44.1M [00:00<00:00, 68.6GB/s]\n",
      "Compute Image Embeddings 2D: 100%|██████████| 1/1 [00:00<00:00,  1.82it/s]\n",
      "Initialize instance segmentation with decoder: 100%|██████████| 1/1 [00:00<00:00,  7.50it/s]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<tr>\n",
       "<td>\n",
       "<img src=\"\"></img>\n",
       "</td>\n",
       "<td style=\"text-align: center; vertical-align: top;\">\n",
       "\n",
       "<table>\n",
       "<tr><td>shape</td><td>(256,&nbsp;256)</td></tr>\n",
       "<tr><td>dtype</td><td>int32</td></tr>\n",
       "<tr><td>size</td><td>256.0 kB</td></tr>\n",
       "<tr><td>min</td><td>1</td></tr><tr><td>max</td><td>39</td></tr><tr><td>n labels</td><td>39</td></tr>\n",
       "</table>\n",
       "\n",
       "</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "StackViewNDArray([[ 6,  6,  6, ...,  5,  5,  5],\n",
       "                  [ 6,  6,  6, ...,  5,  5,  5],\n",
       "                  [ 6,  6,  6, ...,  5,  5,  5],\n",
       "                  ...,\n",
       "                  [39, 39, 39, ..., 37, 37, 37],\n",
       "                  [39, 39, 39, ..., 37, 37, 37],\n",
       "                  [39, 39, 39, ..., 37, 37, 37]], dtype=int32)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load the Segment Anything for Microscopy model.\n",
    "predictor, segmenter = get_predictor_and_segmenter(model_type=\"vit_b_lm\")\n",
    "\n",
    "# Run automatic instance segmentation (AIS) on our image.\n",
    "instances = automatic_instance_segmentation(\n",
    "    predictor=predictor,\n",
    "    segmenter=segmenter,\n",
    "    input_path=image,\n",
    ")\n",
    "\n",
    "# Visualize the instance segmentation result.\n",
    "stackview.insight(instances)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22f0ad18-08fa-4644-bc18-849ac5ab4e2c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2025-04-25T11:29:45.979587Z",
     "iopub.status.busy": "2025-04-25T11:29:45.978859Z",
     "iopub.status.idle": "2025-04-25T11:29:46.410160Z",
     "shell.execute_reply": "2025-04-25T11:29:46.409338Z",
     "shell.execute_reply.started": "2025-04-25T11:29:45.979550Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"\" style=\"width: 256px;\"></img>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can also quickly show the instance segmentation overlay using an animated curtain with 'stackview'.\n",
    "stackview.animate_curtain(image, instances)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7b95caa-9966-40ef-90c0-d3234fe9ff70",
   "metadata": {},
   "source": [
    "## What next?\n",
    "\n",
    "Question: Are you happy with the segmentation on your data?\n",
    "1. If your answer is \"YES\", looks like you are good to go with your segmentations. Feel free to test it on your data.\n",
    "2. If you think there are missed out objects / incorrect segmentations, and there is room for improvement:\n",
    "    1. We recommend you to use our napari-based [annotation tool](https://computational-cell-analytics.github.io/micro-sam/micro_sam.html#annotation-tools) to improve your segmentations.\n",
    "    2. If you want to improve the automatic segmentation quality, we recommend you to finetune the default SAM models OR the `µsam` models, based on your initial assessment of the segmentation quality of both models.\n",
    "3. If you think the models perform poorly, we recommend you to finetune the default SAM models on your custom dataset. There are two situations here:\n",
    "    1. In case you do not have any ground-truth labels, we recommend generating the desired labels using our [annotator tool](https://computational-cell-analytics.github.io/micro-sam/micro_sam.html#annotation-tools)\n",
    "    2. In case you do have ground-truth labels, we suggest going ahead with finetuning already (you can choose to use the [python scripts](https://github.com/computational-cell-analytics/micro-sam/blob/master/examples/finetuning/finetune_hela.py) or the [tool](https://computational-cell-analytics.github.io/micro-sam/micro_sam.html#finetuning-ui) or the [CLI](https://computational-cell-analytics.github.io/micro-sam/micro_sam.html#using-the-command-line-interface-cli) for this.)\n",
    "\n",
    "[Here](https://github.com/computational-cell-analytics/micro-sam/blob/master/notebooks/sam_finetuning.ipynb) is the link to the notebook on how to get quickly started with finetuning on your custom dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54578c7f-83ba-4a20-aadb-5f609174769e",
   "metadata": {},
   "source": [
    "*This notebook was last ran on May 5, 2025*"
   ]
  }
 ],
 "metadata": {
  "kaggle": {
   "accelerator": "gpu",
   "dataSources": [],
   "dockerImageVersionId": 31011,
   "isGpuEnabled": true,
   "isInternetEnabled": true,
   "language": "python",
   "sourceType": "notebook"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
